DOCUMENT RESUME 

ED 381 088 HE 028 226 



AUTHOR 
TITLE 

PUB DATE 
NOTE 



PUB TYPE 



Cheung, K. C. 

Assessing Quality of Learning in Higher Education: 
Methods, Models and Perspectives., 
Jul 94 

13p.; Paper presented at the International Conference 
on Assessing Quality in Higher Education (6th, Hong 
Kong, July 19-21, 1994) . 

Speeches/Conference Papers (150) — Viewpoints 
(Opinion/Position Papers, Essays, etc.) (120) 



EDRS PRICE • 
DESCRIPTORS 



IDENTIFIERS 



MF01/PC01 Plus Postage. 

College Curriculum; '''College Instruction; '-College 
Outcomes Assessment; Educational Philosophy; 
Educational Principles; Elementary Secondary 
Education; Evaluation Methods; Faculty Development; 
Foreign Countries; Formative Evaluation; Higher 
Education; '''Learning Theories; -'Student Evaluation; 
"Teaching Models 
Authentic Assessment 



ABSTRACT 

This paper describes paradigms for research into 
school learning conducted at the primary/secondary and tertiary 
levels. Paradigms discussed include the Process-Product paradigm, the 
Mediating Process paradigm, the Classroom Ecology paradigm, J. 
Carroll's Model of School Learning, B. Bloom's M.istery Learning 
model, and the multilevel causal modeling of the 1980s. At the 
tertiary level, status attainment models describe processes by which 
family status and parent education are converted into occupational 
status through educational attainment. An emergent trend has been 
witnessed in integrating assessment and instruction for quality 
classroom processes and learning outcomes. This trend involves 
changing views of educational indicators and school improvement, 
development of a humanistic and cons t ruct ivi s t curriculum theory and 
practice, and a shift toward meaningful and authentic measurement. In 
order that the quality of learning in higher education can be better 
monitored, educators are urged to pay particular attention to the 
interrelationships between assessment and instruction. Specifically, 
students should be engaged in deep processing of learning tasks, 
teachers should deploy formative assessment for charting learning 
progress, and staff development programs should emphasize pedagogical 
and assessment expertise grounded in a sound theory of curriculum and 
guided by a viable multilevel conceptual model of learning. (Contains 
39 references.) (JDD) 
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A system of higher education most often aim at high standard and 
exceptional quality which may be evaluated in terms of the 
ability levels of the entering students, the qual it ication and 
research output of the staff, and employabi 1 ity and subsequent 
achievement of the graduates. Status attainment models have been 
deployed for this tjpe of evaluation (see Eager lind, 1985 for 
examples). Compared with education at the elementary levels, 
higher education is a lot more expensive. Therefore, efficient 
deployment of human, material and time resources are needed in 
order to meet manpower needs and render community services (see 
Sizer, 1992 for a discussion of accountability and autonomy in 

higher education). Unfortunately, this model of higher education r 

and the quality of its output are increasingly being challenged, 

for example, as the Hong Kong tertiary education system increases 

its intake from 5 to 18 percent of the age group (see Adelman, 

1992 for a discussion of diversity and quality assurance in 

higher education). This challenge is brought about as the 

tertiary teaching staff while recruited because of their academic 

prowess and research competence may not be conversant with the 

principles of effective teaching and learning for the 

heterogeneous group of students. There is a danger that deep 

learning gradually gives way to shallow learning. Consequently, 

there >s a need for staff development programs to enhance the 

pedagogical expertise of the tertiary teaching staff. In order 

to improve instructional effectiveness, formative and summative 

evaluation of the quality of the learning processes and products 

with reference to model of learning is required (see Banta, 

1992, for a review of formative and summative assessment of . 
student achievement in general education and major fields at the 

higher education level). Alternative assessments of the quality , 
of learning (e.g. portfolios, computer simulations of real-world 
problems, hands-on science problems) firmly rooted in a 
humanistic construct ivist theory of knowing are also needed (see 

Harnisch and Mabry, 1993 for a discussion). ( 

With these thoughts in mind, this paper seeks to describe ( 
some emerging paradigms for research into school learning j 
conducted at the primary and secondary levels in the past three "permission to reproduce this 
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decades. It is hoped that these conceptual learning models found 
useful at the elementary levels, together with the status 
attainment models at the tertiary levels, can be of use to inform 
how quality learning at the tertiary levels may be understood and 
evaluation undertaken. After contrasting two conceptions of a 
school which have bearings upon the relationships • between 
educational indicators and school improvement, and asserting the 
need for a humanistic and construct ivist philosophy in curriculum 
theory and practice, as well as commenting on meaningful 
measurement as an emergent trend, this paper concludes that in 
order for quality learning in higher education be secured, there 
is a need for deep cognitive processing in our students, along 
with the promotion of formative assessments by the teachers. 



Paradigms for Research into School Learning 

Conceptualizations of school learning need very clear concepts of 
what teaching is and how it occurs. The former is a 
philosophical question underpinning the processes it involves. 
Hirst's (1975) analyses showed that the intention of all teaching 
activities is that of bringing about learning. The aim of 
learning is to achieve some end s^ate. Specific teaching 
activities must clearly express what is to be learned and take 
place at a level where the student can take on what is intended 
to be learned. Successful teaching would seem to be simply 
teaching which does in fact bring about the desired learning. 
However, criteria of good teaching are nevertheless difficult to 
discern. 

Doyle (1977) explicated three paradigms for. research into 
teaching. The Process-Product paradigm relates teacher variables 
directly to effectiveness indicators such as student learning 
outcomes. It contains few explicit explanatory principles to 
guide the selection of variables or the interpretation of 
results. The Mediating Process paradigm focuses directly on the 
implicit human mental processes that mediate instructional 
stimuli and learning outcomes. Teacher behaviours and 
instructional materials influence learning outcomes only to the 
extent that they activate information-processing responses which 
determine what a student learns. The Classroom Ecology paradigm 
focuses on mutual relations amongst environmental demands and 
human responses in natural classroom settings. One main contrast 
between the latter two paradigms on task performance is that 
Mediating Process research focuses on variations in abilities to 
process subject matter as a source of differences in student 
learning outcomes, whereas the Classroom Ecology approach calls 
attention o the appropriate skills necessary to undertake 
classroom tasks. 

A paradigm, according to Khun (1970), is an implicit 
framework that defines legitimate problems, methods and solutions 
for a research community during a certain period of time. During 
the past three decades, educational research was conducted under 
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the guidance of the above-mentioned three paradigms, proceeding 
from the Process-Product paradigm toward the Mediating Process 
i\nd Classroom Ecology paradigms, as the problem sets and the 
methods used for tackling them changed. Carroll's tradition of 
time and school learning was particularly influential and hence 
deserves careful scrutiny (see Carroll, 1989 for- a review). 
Carroll's thesis is that time is important in school learning 
since the teacher's work consists of the management of students' 
attention over time and over a curriculum. Carroll's (1963) 
leading paper, A Model of School Learning, represents an attempt 
to give a unified perspective on the types of basic variables and 
their inter-relationships that are proposed as affecting a 
student's degree of achievement. This is formally expressed as a 
function of the ratio of the time actively engaged in learning 
(expressed in terms of aptitude, perseverance, and opportunity to 
learn) to the time that the student would need in view of his or 
her aptitude, ability to understand instruction, and the quality 
of instruction offered. 

Bloom's (1974) mastery learning model was an adaptation of 
Carroll's model exchanging variations in student learning for 
variations in time needed for students to learn. Bloom (1976) 
later presented a new formulation of mastery learning principles, 
of which student-entry cognitive and affective characteristics, 
and quality of instruction (i.e. cues, reinforcement, feedback, 
correctives) facilitate active participation (i.e. t ime-on-task) , 
in order to account for the learning outcomes, including improved 
learning rates. The concept of student's t ime-on--task was 
further elaborated by Rosenshine and Berliner (1978) into 
Academic Learning Time, which refers to the time during which the 
student is actively engaged in learning tasks that are somewhat 
challenging and on which a high. success rate can be attained. 
Thus, it is not solely the amount of the time devoted to a 
learning task that counts but also the way in which the time is 
utilized. One principle of effective pedagogy recognized by the 
educators is that teachers should direct and sustain the 
attention of their students. They must select and assign 
appropriate tasks, engage their students in the process of 
achieving the assigned tasks, and keep the students on-task 
(Anderson, 1984). 

Carroll's tradition pievailed into the 1970s, taking a 
structural • approach in the form of causal models of school 
learning toward the end of that decade. The empirical meta- 
analyses made in the early 1980s summarized the research findings 
and helped clarify and understand the meanings and inter- 
relationships of the constituent constructs in the earlier 
conceptual models of school learning (e.g. Fraser, Walberg, Welch 
and Hattie, 1987). Moving toward the end of the 1980s, advances 
in cognitive science research and the adoption of a socio- 
psychological approach to the study of learning environments have 
made the learning models more comprehensive than before - they 
are not only macroscopic in perspective but also microscopic and 
task-specific centring on the cognitive and affective processes 
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of the learners (see Figure 1 and 2 for a multilevel conceptual 
model of school learning; see also Cheung, 1993a, pp. 248-252 for 
an analysis of this model's salient characteristics). 
Development o: multilevel causal modelling as a statistical tool 
for analyzing process-product influence networks within, between, 
and across the different levels of the data hierarchy is also 
actively underway in the past decade (e.g. Cheung, Keeves, Sell in 
and Tsoi, 1990). Fine-grained qualitative analyses of learning 
tasks and the teaching and learning processes are also abundant 
in the literature (see Walberg, 1990 for an evaluation of this 
knowledge base; see also Cheung and Taylor, 1991; Cheung, 1994a 
for design principles of learning tasks). 

«Insert Figures 1 and 2 about here» 

At the tertiary level, status attainment models may be used 
to describe the processes by which family status and parent 
education are converted into occupational status through 
educational attainment, which is viewed as a mediator between 
social background and occupational attainment (e.g. see Blau and 
Duncan, 1967 for an early attempt from the life course 
perspective to examine status attainment; see Fagerlind, 1985 for 
a review). Later, there was a tendency for the status attainment 
models to include more background and process variables, such as 
cognitive ability and quality of schooling, as well as to 
introduce interactive effects of background and process variables 
over and above their combined additive effects. This type of 
research typically establishes the significance of formal 
education mediating between family background and occupational 
status attainment, although there is a tendency for educational 
effects to decline in importance over time. Consequently, in an 
egalitarian society, quality learning processes and outcomes in 
higher education are instrumental for attaining high status lor 
the graduates, explaining the quest for assessing the quality of 
learning in higher education during the past decades. 



Integrating Assessment and Instruction - An Emergent Trend 

The past decade witnessed an emergent trend of integrating 
assessm it and instruction for quality classroom processes and 
learning outcomes. There have been changing views regarding the 
kind and role of educational indicators for the purposes of 
school improvement and an acknowledgement of a humanistic and 
construct ivist view of curriculum theory and practice. Congruent 
with these schooling and curriculum perspectives, which emphasize 
the significance of the interflow of meanings and the effects of 
learning contexts for personal development and school 
improvement, emerged the recent trend of meaningful and authentic 
measurement of learning processes and products. These trends are 
further elaborated below. 



A. Changing Views of Educational Indicators and School 
Improvement 

Byrk and Hermanson (1993) contrasted two conceptions of a school 
which have implications for the design of educational indicators 
for the purposes of school improvement. The first conception of 
a school is that it is viewed as a formal, bureaucratic 
organization whose constituents are characterized in terms of 
their defined roles, functions and structures, with limited 
personal discretion and an unambiguous accountability system. 
This rational-bureaucratic perspective results in educational 
indicators being designed mainly to tap critical resources, 
processes and outcomes so that, the school operations can be 
parsimoniously represented as a production function (for examples 
in different higher education systems, see Cave and Hanney, 
1992). The administrative staff is then able to exercise 
rational judgements on ways of school improvement and system 
restructuring. 

The second conception of a school is that it is viewed as a 
social system where personal interactions are primary and that 
system reform often requires changing the values and tacit 
understandings that ground these interactions. This personal- 
cominunitarian perspective results in indicators being 
comprehended as information flows, communication channels, and 
feedback loops of personal meanings and intentionality. Prudent 
use of indicators require that such information be selected and 
recontextual ized from a conceptual model of the operative 
educational system (preferably having a multilevel formulation 
within a cultural context) to particular cases or situations in 
order to understand adequately its salience for systems 
improvement. Hence, the multilevel conceptual model shown in 
Figure 1 and 2 should as far as practicable be analyzed and 
interpreted with this perspective in mind. 

B. Humanistic and Construct ivist Curriculum Theory and Practice 

Humanistic education seeks to increase students' self-direction 
and curiosity and help them take responsibility for determining 
what they need and want to learn (Gage and Berliner, 1992). 
Moreover, learning how to feel is as important as learning how to 
think, and students' own evaluation is the only meaningful 
judgement of their work. From the construct ivist perspective, 
learning is a process concerning the construction of meanings. 
Via social justification, the personal knowledge constructed is 
being negotiated to attain the status of publ icly-mediated 
knowledge (for a discussion, see Cheung and Taylor, 1991). Since 
knowledge is created in response to the needs of the society, it 
is important that students can understand the changing needs of 
the communities where they live and how knowledge is 
progressively being developed in different cultural contexts. 
They should be encouraged to pose and solve personally-relevant 
problems, aware of the nature and meanings of the problems 
perplexing them, and investigate controversial issues of societal 
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concern. Their enquiries need to be based on multi-level and 
multi-angled perspectives and their choices and value 
orientations need to be personally carefully evaluated. Most 
important of all, students should be educated to deploy global 
resources and cultural heritage wisely (see Cheung, 1993c & 1994b 
for a discussion of sustainable living Earth and notions of 
sustainable development). Therefore, learning outcomes encompass 
not only facts and skills, but also feelings, values and meanings 
constructed as a result of organizing and making sense of their 
everyday and learning experiences. 

C. Meaningful and Authentic Measurement as an Emergent Trend 

During the 1990s, there has been a shift of norm-referenced 
formal assessment of individual students (e.g. the contrived 
standardized achievement tests) toward criterion-referenced 
authentic classroom assessment (e.g. portfolios of student work 
across time and task contexts within a classroom setting) and 
performance assessment (e.g. investigative and problem-solving 
tasks in real-world settings) (see Wiggins, 1992 for a 
discussion). Authentic classroom assessment and performance 
assessment acknowledge students' cultural and societal 
diversities by allowing alternative ways of solving problems and 
accomplishing tasks. These new modes of assessment, often with 
teachers and peers as participant observers and evaluators, focus 
on what students can do in a holistic manner or on whether they 
can meet the required standard, rather than, on how well they can 
do relative to other students. Therefore, it has been suggested 
that scaffolding in the form of teacher instruction, peer 
interaction, and cooperative learning is a legitimate and 
expected part of the assessment process, highlighting the close 
interplay between assessment and instruction (Garcia and Pearson, 
1994; see also Feuerstein, 1979 for his notion of dynamic 
assessment ) . 



Discussion 

In order that the quality of learning in higher education can be 
better monitored than hitherto, teaching staff of institutes of 
higher learning need to pay particular attention to the inter- 
relationships between assessment and instruction (see Glaser and 
Silver, 1994 for a review of some of these efforts). 
Specifically, students should be engaged in deep processing of 
the learning tasks presented to them (see Figure 2 for the 
various constructs and process variables involved). Furthermore, 
teachers should deploy formative assessment for charting learning 
progress and expressing students' shortcomings and potential. 
This proposal, which has implications on the aims and content of 
the staff development program for the development of pedagogical 
expertise, is elaborated further below. 
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A. Deep, Surface and Strategic Approaches to Learning 

Entwistle (1994) noted that students entering higher education 
are commonly not well-prepared for the different types of 
learning and studying required of them. Rather than the teachers 
organizing the learning experiences for the students through 
careful presentation in packaged form and regular assignments, 
students of higher education have to increasingly assume greater 
responsibility in practising independent studying. Entwistle 
asserted that the three common approaches to learning (i.e. deep, 
surface, and strategic processing of the learning tasks) are each 
supported by a distinctive form of motivation (i.e. intrinsic 
motivation, instrumental mot ivat ion/f ear of failure, and need for 
achievement for the three approaches to learning respectively). 
It has been noted that there is a tendency for students to adjust 
their approaches to learning in response to the different forms 
of assessment (e.g. surface processing for the short-answer and 
multiple-choice questions; deep processing for the open-ended 
essay questions, problem-solving and project work) and what they 
want to gain from the course (i.e. in accordance with their 
academic, vocational, personal, and social orientations of 
study) . 

Deep processing of course content was shown by some research 
to be more successful than surface processing (e.g. Heggarty- 
Hazel and Prosser, 1991). Furthermore, it has been observed that 
when under pressures of tests and examinations, the deep- 
processing learners may revert to surface processing in order to 
alleviate the problem of cognitive overload. Consequently, 
learning styles (e.g. holist vt ' sus serial ist in accordance with 
the two very contrasting ways of perceptual processing and 
meaning construction) affect immensely the qualit}' of learning 
processes and outcomes (Pask, 1976). As an example, students 
learn not only faster but also more fully when their learning 
styles match with the requirements and features of the learning 
materials. Since cognitive processing takes time and students' 
ideas do not emerge in continuous form but tend to burst in 
discrete form with gaps in between, wait time before and during a 
student's response to questions is also an important quality 
factor because this is when meanings are constructed and 
negotiated (Tobin, 1984). 

B. Formative and Summative Assessment by Teachers 

Formative assessment by the teachers is central to good teaching 
and learning (Black, 199J). This is because the formative 
assessment information can be immediately used by both the 
students and teachers to modify their work in order to make it 
more effective (see the feedback loops in Figure 1 and 2). Some 
form of differentiated teaching may be required in order to cater 
for the heterogeneity in the learning needs of the students. 
Summative assessment, however, is continued to be essential for 
the certification and accountability functions. 
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Formative assessment, when integrated with the instructional 
processes guided by a sound conceptual model of learning and 
rooted in a humanistic/constructivist curriculum perspective, 
promotes effective learning. The scope of classroom assessment 
may be further broadened so that assessment is no longer limited 
to the examination of decontextual ized isolated facts or skills 
using timed paper and pencil tests. Instead, there can be more 
frequent use of observations of student work in progress, 
performance assessments of authentic learning tasks presented in 
various contexts, and systematic recording against criteria which 
are results of negotiations between teachers and students. 
Consequently, criterion-referenced assessment feedback, 
characteristic of formative assessment and central to the 
persona 1-communi tar i an conception of a school, help better 
monitor student progress and express potentials and shortcomings 
than that of the high-stakes summative assessment common ly 
associated with schools taking the rational-bureaucratic 
perspective. 

With reference to a model of progression and development for 
drawing inferences and recommending paths of learning within a 
clearly specified domain of study, the construct validity of the 
formative assessment feedback is increased when inferences and 
actions can be -shown to improve the quality of learning and 
satisfy the needs of the students (see Cheung and Taylor, 1991 
for a viable model of human progression and development). 
Furthermore, self-assessment by students and peer appraisal 
amongst students, with a distinctive advantage of holding the 
students responsible for their ov/n learning progress and 
determining success criteria and mastery standards, may be 
included as viable formative assessment tools as well (see 
Falchikov and Boud, 1989 for a review). 

In sum, this paper argues for the need for the students in 
higher education to take a deep approach to learning and for the 
teachers to deploy formative assessment of the learning processes 
and products. It highlights the importance of communication 
channels and feedback for institutional improvement. Staff 
professional development programs should not only emphasize 
academic prowess and research competence but also pedagogical and 
assessment expertise which should be grounded in a sound theory 
of curriculum and guided by a viable multilevel conceptual model 
of learning. 
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Figure 1 



Conceptual Model of School Learning - Institutional Level 
(from Cheung, 1992; first appeared in Cheung, 1989, p. 62) 

School institution/organisation 



Ecology & School processes (socitl system & culture) 

milieu 

Classroom Instructional unit 




Figure 2 

Conceptual Model of School Learning - Learner Level 
(from Cheung, 1993b; first appeared in Cheung, 1989, p. 72) 




Note: 



ERIC 

cjMaiMBBimiu 



:: r~\ Comprehension, a sum.vtary mediating construct for task perception, cognitive and 
^/ memory process, is the anleccndcnl of task mastery. 

j-A Feedback, tangible or intangible experience, to the learning tasks and cognitive 
t-y processing system. 
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